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1  Introduction 

Solving  the  PFA  and  PFR  for  non-normally  distributed  measurements  errors  is  useful  because 
sometimes  the  data  will  not  follow  a  normal  distribution.  When  this  is  the  case,  different 
distribution  curves,  such  as  uniform  or  triangular,  must  be  used  to  calculate  the  PFA  and  PFR. 

This  paper  will  describe  and  review  how  to  theoretically  calculate  the  PFA  and  PFR  given  a 
specific  distribution  and  the  associated  joint  probability  density  function  (PDF).  Then,  assuming 
uniformly  distributed  measurement  errors,  we  will  try  to  find  a  formula  for  its  PFA  and  PFR. 
After  this  is  done,  we  will  simulate  and  examine  the  calculation  for  uniformly  distributed 
measurement  errors  using  Visual  Basic  for  Applications  (VBA),  in  hopes  of  confirming  our 
findings  from  our  theoretical  calculations  of  the  PFA  and  PFR. 


2  Probability  of  False  Accept  and  Probability  of  False  Reject 

A  False  Accept  is  defined  as  the  event  when  a  unit  under  test  (UUT)  is  measured  to  be  outside 
calibration  tolerance  limits  but  the  difference  of  the  observed  measurement  results  of  the  UUT 
and  the  Calibration  Standard  (CalStd  or  CAL)  is  within  acceptance  limits. 

A  False  Reject  is  defined  as  the  event  when  a  UUT  is  measured  to  be  within  calibration 
acceptance  limits  but  the  difference  between  the  observed  measurement  results  of  the  UUT  and 
the  CalStd  is  outside  tolerance  limits. 


Hence,  the  PFA  is  defined  as  the  probability  of  making  an  acceptance  decision  when  the  UUT  is 
observed  and  reported  to  be  in-tolerance  but  is  actually  out-of-tolerance.  The  PFR  is  defined  as 
probability  of  making  a  rejection  decision  when  the  UUT  is  observed  and  reported  to  be  out-of- 
tolerance  but  is  actually  in-tolerance. 

Here  are  some  standard  notation  and  variables  that  will  be  used: 


^UUT 

=  The 

^CAL 

=  The 

d 

=  The 

-L 

=  The 

L 

=  The 

-A 

=  The 

A 

=  The 
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When  deriving  a  formula  for  the  PFA  or  PFR,  we  use  elementary  probability  theory  to  describe 
where  our  d  and  our  eyuT  lie.  After  examining  the  definitions  of  the  PFA  and  PFR  we  can  easily 
see  that: 


PFA  =  Prob{(-A  <  d  <  A)  AND  [(euur  >  QOR  {buut  < 
and 

PFR  =  Prob{(-L  <  eyuT  ^  L)  AND  [(d  >  A)  OR  (d  <  -A)]} 

With  these  definitions  for  the  PFA  and  PFR,  only  the  joint  PDF  of  d  and  e^yj-is  needed  to  have  a 
complete  equation  for  the  PFA  and  PFR.  Sometimes  the  joint  PDF  will  be  assumed  or  have  to  be 
derived. 

The  joint  PDF  is  defined  as: 


/ (d,  euuj)  —  /(d  I  euuj)  ■  / 

where 


f{d\euuT)  =  The  conditional  PDF  of  d  given  euuT 
fi^uuT^  ~  The  PDF  of  SyyT’ 

With  this  information  the  PFA  and  PFR  become: 


r  OO  pA  r — L  r /i 

P  FA  =  If  (d,  euuj)  dddcjjjj'p  “hi  I  ^uuT^  dddcjju'j' 

Jl  J —A  J—oo  J —A 


-L  rA 


and 


r-A 


p  L,  ^  uo  p  p  — 

PFR  =  I  I  f  (^d,euuj)  dddeuuj  I  I  fid.euuj^  dddeyijx 

J-L  'a  J-L  J-oo 


4  Uniform  Distribution 

The  Uniform  Distribution  has  a  constant  PDF  that  forms  a  rectangle.  The  PDF  is  given  by: 

r  1 


f(x)  =  fe'  -a  <x  <  a 


0, 


otherwise 


with  variance  and  uncertainty  as  follows: 


variance{x)  =  — 
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j  CL 

uncertainty  ipc)  =  ^J  varianceipc)  =  — 

V3 


The  PDF  for  the  Uniform  Distribution  can  also  be: 


i  0, 


a  <  X  <  b 
otherwise 


With  variance  and  uncertainty  as  follows: 


varianceix)  = 


(b  —  a)^ 
12 


I - ; -  U  —  U 

uncertainty  {x)  =  varianceix)  =  _ 

yl2 

5  Monte  Carlo  Simulation  of  Uniform  Distribution 

Through  VBA,  a  simulation  can  be  developed  that  calculates  both  the  PFA  and  the  PFR  of 
measurements  errors  with  a  Uniform  Distribution.  We  will  define  the  two  functions  that  calculate 
the  PFA  and  PFR  as  USimFa  and  USimFr,  respectively.  Each  of  these  functions  will  take  the 
same  5  inputs: 


b 

c 

A 

L 

simCnt 


The  UUT  limit. 

The  CAL  limit. 

The  tolerance  limit. 

The  acceptance  limit. 

The  number  of  times  the  simulation  will  repeat  itself. 


Within  each  function,  there  are  several  variables,  which  will  be  used  in  each  iteration  to  calculate 
the  number  of  times  that  the  difference  of  the  measurement  error  of  the  UUT  and  CalStd  (d)  and 
the  measurement  error  of  the  UUT  (e^^f)  lie  inside  or  outside  the  correct  intervals  necessary  to 
be  a  false  accept  or  false  reject.  To  insure  each  simulation  is  random,  there  will  be  a  Randomize 
(Rnd)  function  that  randomly  chooses  a  number  between  0  and  1,  which  is  used  to  calculate  the 
euuTr.  6cal>  and  d.  Each  time  the  correct  requirements  are  satisfied  for  PFA  and  PFR,  the 
SumFa  or  SumFr  will  be  increased  by  1 .  Then  the  loop  will  start  again  with  a  new  random 
number.  Once  the  simCnt  is  reached,  the  PFA  or  PFR  will  be  calculated  by  dividing  SumFa  or 
SumFr  by  simCnt  to  obtain  PFA  or  PFR,  respectively.  The  VBA  code  for  this  simulation  is  as 
follows: 
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Option  Explicit 

Function  USimFa  (b  As  Double,  c  As  Double,  A  As  Double, 
Double,  simCnt  As  Long)  As  Double 
'b  is  OUT  limit 
'c  is  CAL  limit 
'A  is  acceptance  limit 
'L  is  specification  limit 

Dim  eUUT  As  Double 
Dim  eCAL  As  Double 
Dim  d  As  Double 
Dim  FaAvg  As  Double 
Dim  SumFa  As  Double 
Dim  Fa  As  Long 
Dim  i  As  Long 
SumFa  =  0 

For  i  =  1  To  simCnt 

'initialize  random  number  generation 
Randomize 

eUUT  =  (Rnd()  *  2  *  b)  -  b 
eCAL  =  (Rnd()  *  2  *  c)  -  c 
d  =  eUUT  -  eCAL 

If  (d  >=  -A  And  d  <=  A)  And  (eUUT  >  L  Or  eUUT  <  -L) 
Fa  =  1 

Else 

Fa  =  0 
End  If 

SumFa  =  SumFa  +  Fa 

Next 

USimFa  =  SumFa  /  simCnt 
End  Function 


Option  Explicit 

Function  USimFr (b  As  Double,  c  As  Double,  A  As  Double, 
Double,  simCnt  As  Long)  As  Double 


L  As 


Then 


L  As 
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'b  is  OUT  limit 
'c  is  CAL  limit 
'A  is  acceptance  limit 
'L  is  specification  limit 

Dim  eUUT  As  Double 
Dim  eCAL  As  Double 
Dim  d  As  Double 
Dim  FaAvg  As  Double 
Dim  SumFr  As  Double 
Dim  Fr  As  Long 
Dim  i  As  Long 
SumFr  =  0 

For  i  =  1  To  simCnt 

'initialize  random  number  generation 
Randomize 

eUUT  =  (Rnd()  *  2  *  b)  -  b 
eCAL  =  (RndO  *  2  *  c)  -  c 
d  =  eUUT  -  eCAL 

If  (eUUT  >=  -L  And  eUUT  <=  L)  And  (d  >  A  Or  d  <  -A)  Then 
Fr  =  1 

Else 

Fr  =  0 
End  If 

SumFr  =  SumFr  +  Fr 

Next 

USimFr  =  SumFr  /  simCnt 
End  Function. 

With  these  two  functions  we  can  also  create  a  Subroutine  that  is  assigned  to  a  button,  which 
calculates  the  PFA  and  PFR  from  given  inputs  to  the  Excel  worksheet.  The  Excel  worksheet 
would  look  as  follows: 
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A 

B 

C 

D  1 

1  E 

F 

G  H  1  1  1 

1 

Lower 

Upper 

2 

UUT  Limits 

CAL  Limits 

Calculate  PFA  and  PFR 

Tolerance  Limits 

~Y\ 

Acceptance  Limits 

Simulation  Count 

~ri 

PFA 

PFR 

The  VBA  code  assigned  to  the  button  would  be  the  following: 

Option  Explicit 

Sub  Calculate  0 

Dim  dUUTLimLow  As  Double 
Dim  dUUTLimUp  As  Double 
Dim  eUUTLimLow  As  Double 
Dim  eUUTLimUp  As  Double 
Dim  TolLow  As  Double 
Dim  TolUp  As  Double 
Dim  AcceptLow  As  Double 
Dim  AcceptUp  As  Double 
Dim  simCnt  As  Long 

Dim  rnginput  As  Range 
Set  rnginput  =  Range ("D2") 

dUUTLimLow  =  rnginput (1,  1) 
dUUTLimUp  =  rnginput  (1,  2) 
eUUTLimLow  =  rnginput (2,  1) 

eUUTLimUp  =  rnginput (2,  2) 

TolLow  =  rnginput  (3,  1) 

TolUp  =  rnginput  (3,  2) 

AcceptLow  =  rnginput (4,  1) 

AcceptUp  =  rnginput (4,  2) 
simCnt  =  rnginput (5,  1) 

Dim  rngOutput  As  Range 
Set  rngOutput  =  Range ("D8") 

Dim  dPFA  As  Double 

dPFA  =  USimFa (dUUTLimUp,  eUUTLimUp,  TolUp,  AcceptUp,  simCnt) 
rngOutput (1,  1)  =  dPFA 
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Dim  dPFR  As  Double 

dPFR  =  USimFr (dUUTLimUp,  eUUTLimUp,  TolUp,  AcceptUp,  simCnt) 
rng0utput(2,  1)  =  dPFR 

End  Sub . 

6  Simulation  Results 

Here  are  some  results  of  calculating  the  PFA  and  PFR  with  different  limits. 


A 

B 

C 

D 

E 

F 

G 

H 

1 

1 

Lov»/er 

Upper 

UUT  Limits 

-m 

10 

Calculate  PFA  and  PFR 

~3 

CAL  Limits 

-5 

5 

4 

Tolerance  Limits 

-4 

4 

5 

Acceptance  Limits 

-6 

6 

6 

Simulation  Count 

1000000 

7 

S 

PFA 

0.044301 

S 

PFR 

0.253253 

A 

B 

C 

D 

E 

F 

G 

H 

1 

1 

Lower 

Upper 

UUT  Limits 

-1 

1 

Calculate  PFAandPFR 

~3~ 

CAL  Limits 

-0.25 

0.25 

4 

Tolerance  Limits 

-1 

1 

~Y~ 

Acceptance  Limits 

-1 

1 

6 

Simulation  Count 

1000000 

7 

S 

PFA 

0.000000 

3 

PFR 

0.065S35 

The  second  simulation  had  the  surprising  result  that  the  PFA  was  exactly  0%.  With  the  Uniform 
Distribution,  100%  of  the  UUT  measurements  errors  should  occur  between  the  limits  of  the 
UUT.  If  these  limits  are  the  same  as  or  less  than  the  tolerance  limits,  no  measurement  errors  will 
ever  be  out-of-tolerance.  As  a  result,  there  will  never  be  a  false  accept.  Hence,  the  PFA  would 
be  0%,  which  the  simulation  confirms 

Also,  in  this  situation,  no  UUT  measurement  errors  will  ever  be  out-of-tolerance.  Therefore,  all 
observed  out-of-tolerances  will  be  false  rejects  caused  by  the  error  in  the  calibrator.  This  means 
that  the  PFR  will  not  be  0%,  which  the  simulation  confirms. 

It  should  be  noted  here,  though,  that  this  result  calls  into  question  whether  the  Uniform 
Distribution  is  useful  for  assessing  calibration  decision  risk. 
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7  Conclusions 

Although  finding  a  closed-form  solution  for  a  measurement  error  that  follows  the  Uniform 
Distribution  amounts  to  finding  the  correct  joint  PDF  is  difficult,  writing  a  simulation  with  VBA 
that  simulates  the  Uniform  Distribution  was  quite  easy.  An  attempt  to  find  the  joint  PDF  of  two 
uniform  random  variables  was  not  successful.  However,  the  simulation  gives  accurate  and 
valuable  information  for  assessing  the  risk  associated  with  measurement  errors  following  the 
Uniform  Distribution. 

Generally,  risk  assessment  is  used  in  situations  where  genuine  out  of  tolerance  results  are 
possible.  Though  the  Uniform  Distribution  is  a  good  model  for  some  kinds  of  measurement 
error  (e.g.,  resolution),  it  would  be  rare  that  a  calibration  measurement  error  would  be  bounded 
completely  by  known  limits  as  required  by  this  distribution.  So,  while  the  Uniform  Distribution 
can  be  used  to  get  conservative  measurement  uncertainty  estimates,  it  is  suggested  that  this 
distribution  is  generally  not  appropriate  for  assessing  measurement  risk. 
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